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1.  Introduction 


This  report  describes  the  last  phase  of  our  work  under  grant  DAAL03-91-G-0312.  In  this 
phase  we  have  applied  some  of  the  concepts  and  devices  developed  earlier  to  the 
construction  of  a  novel  optical  processor.  The  system  described  here  extracts  by  its  own 
meaningful  temporal  features  from  its  input  environment. 

A  temporal  feature  extractor  detects  the  main  characterizing  temporal  features  within  a 
more  or  less  structured  signal.  The  task  it  accomplishes  is  better  understood  by  using  a 
simple  example.  Imagine  to  tune  across  a  short-wave  radio  band.  It  will  be  easy  to 
recognize  a  channel  that  carries  Morse  code;  even  without  knowing  the  code.  That  is 
because  Morse  code  consists  of  a  simple  set  of  temporal  features  (a  dot,  a  dash  and  two 
pause  lengths),  and  a  Morse  signal  is  characterized  by  their  repeated  occurrences.  It  does 
not  take  very  long  for  the  brain  to  identify  the  features  as  the  dominant  content  of  the 
received  signal.  The  task  of  our  feature  extractor  is  just  this:  to  discover  on  its  own  the 
dominant  features  in  a  temporal  signal  characterized  by  repetitive  entities  [1, 2].  This  task  is 
a  precursor  to  the  more  complex  processing  required  for  the  self-organized  feature 
extraction  and  recognition  of  audio  and  sonar  signals. 

In  a  previous  project  phase  we  did  implement  an  optical  classifier  for  the  recognition 
of  acoustic  signals  [3]  which  was  based  on  a  time-delay  neural  network  architecture  [4]. 
This  system  required  the  presence  of  a  teaching  signal.  Unlike  the  acoustic  processor,  the 
present  temporal  extractor  is  self-organizing  [5-7]  and  does  not  need  a  teacher.  This 
approach  is  most  helpful  when,  like  in  the  currently  studied  problem,  one  does  not  have  a 
priori  knowledge  of  what  or  when  to  learn  from  the  input  environment. 

2.  Highlights 

A  model  for  temporal  feature  extraction  is  discussed  and  its  optical  hardware 
implementation  is  described  in  detail.  The  extraction  of  features  within  both  binary  and 
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analog  temporal  signals  modulated  on  a  Gaussian  beam  are  demonstrated.  The  separation 
of  two  different  temporal  signals  into  different  output  ports  through  the  competitive 
dynamics  in  the  optical  resonator  system  is  also  successfully  verified. 

3.  Basic  Principles  of  the  Optical  System 

The  extraction  of  temporal  features  can  be  achieved  using  the  optical  architecture,  which 
basic  scheme  is  shown  in  Fig.l.  The  system  contains  two  basic  processing  elements,  the 
gain  crystal  (long-term  memory)  and  the  optical  delay-line  [8]  which  provides  a  short-term 
memory  element.  The  system  operation  makes  use  of  the  time-domain  cooperative 
interactions  provided  by  the  delay  line  in  addition  to  the  nonlinear  photorefractive  dynamics 
in  optical  resonators  [9]. 

In  Fig.l  each  ring  corresponds  to  a  spatial  mode  in  the  optical  resonator.  The  group 
of  modes  share  a  photorefractive  gain  crystal  and  are  arranged  along  the  delay  coordinates 
of  the  optical  delay  line.  At  each  moment  in  time,  the  delay  line  temporarily  stores  the 
optical  field  of  each  mode  (e.g.,  mode  J),  and  later  injects  the  stored  field  into  the  neighbor 
mode  0+1)-  Its  action  can  be  expressed  as 

Ej+i(t)  =  Gj+iSU)  +  - 1),  0=1.  n- 1 )  (1) 

where  Ej(t)  is  the  optical  field  within  mode  j  at  time  t,  Gj  is  a  normalized  grating 
component  in  the  gain  crystal,  and  0</i<l  is  a  numerical  factor  characterizing  the  decay 
along  the  delay  line.  The  temporal  signal  S(t)  is  imposed  on  a  Gaussian  pump  beam  as 
modulation  on  its  amplitude  and/or  phase;  this  serves  as  input  to  the  system.  The  grating  Gj 
in  the  gain  crystal  scatter  the  pump  beam  into  the  resonator  mode;  this  part  of  the  light 
contains  information  about  the  present  input.  In  addition,  each  mode  receives  injected  light 
from  the  delay  line  which  contains  information  about  past  input  signals.  Note  that  for 
clarity  of  presentation  a  spatially  discrete  array  of  n  modes  with  time  delay  t  between  two 
neighboring  modes  has  been  assumed.  However,  this  is  not  necessarily  the  case;  the  modes 
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can  also  be  continuously  distributed  in  space,  as  in  the  experiments  described  later.  Using 
Eq.(l),  the  total  optical  field  Ej(t)  in  the  mode  at  time  t  can  be  written  as 

j  .  . 

Ejit)='^Gi,S(t-jr  +  kt)n^-\  0=1 . n).  (2) 

k=\ 

From  Eq.(2)  it  is  clear  that  given  a  set  of  gratings,  the  optical  field  in  a  mode  is  an  inner 
product  between  the  gratings  (in  space)  and  the  input  signal  (in  time).  Thus,  the  gratings 
can  be  thought  of  as  defining  a  temporal  feature  for  the  system.  In  general,  the  maximum 
time  delay  determines  the  length  of  temporal  features  that  the  resonator  can  be  sensitive  to, 
and  the  bandwidth  of  the  optical  delay  line  defines  the  minimum  time-scale  for  input  signal 
changes. 

The  photorefractive  grating  G  in  the  gain  crystal  starts  initially  from  noise.  It  grows 
when  oscillations  in  the  resonator  begin  to  build  up.  The  above  unidirectional  coupling  in 
the  delay  element  (Eq.  (1))  modifies  the  spatial  structure  of  the  collection  of  all  the  coupled 
modes.  We  refer  to  this  mode  collection  as  a  chronomode.  The  final  equilibrium  structure 
and  the  equilibrium  grating  (G;  ...G^)  are  determined  by  the  temporal  characteristics  of  the 
input  signal  S(t)  of  interest.  In  other  words,  the  gain  crystal  grating  matches  the  particular 
chronomode  spatial  structure  corresponding  to  the  dominant  temporal  features.  The  grating 
can  be  thought  of  as  a  matched  filter,  in  the  sense  that  it  permits  resonator  oscillation  only 
when  the  dominant  feature  is  present  at  the  input.  For  non-dominant  features,  that  is 
features  occurring  only  rarely,  oscillation  is  inhibited. 

Fig.l  shows  a  single  group  of  modes,  that  is  a  single  chronomode.  The  system  can 
contain  multiple  chronomodes,  each  arranged  along  a  different  delay  line.  If  the 
chronomodes  share  a  common  gain  crystal,  each  of  them  can  learn  a  particular  temporal 
feature  of  the  input  environment.  Fig.2  shows  schematically  two  chronomodes  competing 
in  the  central  gain  crystal.  Each  resonator  ring  in  Fig.2  is  expected  to  respond  strongly 
when  its  learned  feature  is  presented  at  the  input. 
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We  should  note  that  the  simple  optical  schemes  described  above  can  be  translated  in 

equivalent  neural  network  architectures  including  time  delays  within  sets  of  output  units. 

Our  analysis  of  the  network  models,  of  which  we  skip  details  here,  has  proven  very  helpful 

to  qualitatively  predict  the  general  behavior  of  the  actual  architectures  under  consideration. 

* 

4.  Hardware  Implementation  and  Experimental  Verification 

We  first  describe  the  set-up  shown  in  Fig.  3  used  to  implement  the  self-organizing 
temporal  feature  extractor.  All  the  beams  shown  are  from  a  single  frequency,  cw  Ar-ion 
laser  (X,  =  514  nm),  and  are  p-polarized.  The  system  consists  of  an  optical  resonator 
formed  between  the  rotating  photorefractive  delay  line  crystal  (BaTiOs  #1)  [8]  and  the 
feedback  mirror  Mi.  The  resonator  is  pumped  by  a  modulated  Gaussian  beam  entering  the 
gain  crystal  (450-cut  BaTiOs,  #3)  which  lies  in  the  image  plane  of  crystal  #1.  Note  that  the 
photorefractive  delay  line  works  in  the  phase-conjugate  configuration,  the  counter- 
propagating  pump  beam  is  generated  by  a  phase  conjugate  mirror  implemented  by  four- 
wave  mixing  in  the  crystal  BaTiOs  #2. 

Each  mode  in  the  resonator  finishes  a  round  trip  by  tracing  the  same  path  twice,  and 
reflecting  twice  at  crystal  #1.  The  optical  phase  is  reproduced  after  one  round  trip,  therefore 
the  resonance  condition  is  maintained  independent  of  the  optical  length  of  the  beam  path. 
The  time  delay  coordinate  of  the  delay  line  is  along  a  cone  [8],  a  short  portion  of  it 
essentially  lie  along  the  direction  peipendicular  to  the  plane  of  Fig.  3.  A  group  of  modes 
distributed  along  this  vertical  direction  defines  a  chronomode  in  the  experiment.  Its 
transverse  profile  is  constrained  by  a  pair  of  slot  apertures  in  the  horizontal  plane.  Fig.3 
shows  two  chronomodes  identified  by  the  differently  dashed  lines.  In  general  the  system 
can  have  many  chronomodes  as  long  as  there  is  sufficient  gain  in  the  pump  crystal  (#3)  so 
that  all  can  be  above  threshold  and  oscillate.  The  vertical  length  of  the  slot  apertures  defines 
the  angular  range  over  which  modes  enter  the  delay  line  and  gain  crystals,  which  effectively 
defines  the  maximum  time  delay  within  the  system. 
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Note  that  the  scheme  of  Fig.  3  is  not  unique.  Other  architectures  have  been 
successfully  tested.  All  of  them  share  the  common  characteristics  of  containing  a  long  term 
memory  gain  crystal  and  a  short  term  memory  rotating  crystal  within  differently  designed 
photorefractive  resonators.  The  scheme  of  Fig.3  was  found  to  be  the  most  robust  because 
amplification  occurs  in  the  gain  crystal  for  light  traveling  in  both  directions  along  the 
resonator  path.  This  decreases  the  oscillation  threshold. 

We  have  performed  experiments  to  verify  the  operation  of  the  signal  processor  using 
binary,  analog  as  well  as  partially  noisy  signals.  We  could  verify  that  the  following  two 
properties  are  satisfied: 

(1)  If  a  certain  temporal  feature  is  clearly  dominant,  a  system  containing  a  single 
chronomode  will  select  that  feature  at  the  expense  of  all  other  non-dominant  ones. 

(2)  In  a  system  with  two  competing  chronomodes,  each  of  two  equally  dominant 
features  is  associated  with  one  of  the  chronomodes.  If  there  is  only  one 
chronomode  an  ambiguous  state  may  result. 

In  the  experiments  a  dominant  feature  (sub-signal)  is  the  one  which  is  presented  most  often 
within  the  input  signal.  By  changing  the  rate  of  occurrence  of  a  selected  sub-signal  the 
input  can  be  deliberately  biased  in  any  wanted  direction.  We  present  below  the  results  of 
two  experiments  which  demonstrate  the  two  above  properties.  Deep  experimental  details 
are  skipped,  they  can  be  found  in  the  enclosed  preprint  of  Ref.  [1]. 

Fig.4  shows  the  results  of  an  experiment  demonstrating  property  (1),  the  learning  of 
the  most-frequently  presented  sub-signal  within  a  single-chronomode  system.  The  phase- 
modulated  binary  signals  defined  in  the  caption  are  used.  Note  that  by  switching  from  +  to 
-  the  pump  beam  gets  opposite  phase  but  keeps  the  same  intensity.  In  Fig.4  the  total 
intensity  of  the  resonator  is  plotted  as  a  function  of  time  as  different  sub-signals  are  applied 
at  the  input.  The  figure  shows  the  equilibrium  state  which  is  reached  after  about  20  learning 
cycles.  In  the  experiment  of  Fig.4a  sub-signal  Si  dominates  S2  by  a  ratio  of  2  to  1.  As 
expected,  in  this  case  the  resonator  response  is  much  stronger  while  Si  is  applied.  In  the 
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opposite  case  (Fig.4b)  S2  dominates  by  2  to  1  and  the  chronomode  responds  now  to  52* 
Note  that,  as  a  sub-signal  is  presented  at  the  input,  the  resonator  response  grows 
continually  from  the  beginning  till  the  end.  This  is  an  interesting  property  of  the  self- 
organized  state.  The  system  can  be  thought  of  as  being  able  to  accumulate  evidence 
continually  over  time,  which  is  somewhat  similar  to  the  human  ability  to  continually 
anticipate  and  make  predictions  as  we  listen  to,  e.g.,  speech  or  music.  Other  types  of  signal 
processors  such  as  time-delay  neural  network  [3, 4]  do  not  posses  this  property. 

Next  we  demonstrate  property  (2).  In  this  experiment  the  resonator  contains  two  pairs 
of  apertures  and  thus  two  chronomodes  can  oscillate.  This  situation  corresponds  to  the 
scheme  of  Fig.  2.  Analog  sub-signals  with  strong  random  components  overlapping  regular 
sinusoidal  functions  arc  used.  The  form  of  the  two  sub-signals  (Signal  1  and  Signal  2)  to 
be  learned  is  shown  in  Fig.5(a).  Fig.  5(b+c)  show  the  response  of  the  two  chronomodes 
in  the  equilibrium  state.  Each  image  corresponds  to  the  resonator  intensity  profile  at  the 
moment  when  a  signal  has  just  been  applied  at  the  input.  The  pictures  are  taken  at  the  place 
of  the  detector  in  Fig.  3  using  a  CCD  camera.  One  sees  that  one  of  the  chronomodes 
responds  strongly  to  Signal  1  and  the  other  to  Signal  2.  Each  chronomode  has  learned  to 
associate  with  a  particular  sub-signal.  Note  that  in  this  experiment  the  basis  for  the 
recognition  is  given  by  the  highly  autocorrelated  sinusoidal  functions  underlying  the  two 
signals.  Completely  noisy  and  unstructured  signals  cannot  be  learned. 

We  have  studied  different  expansion  schemes  for  the  system  of  Fig.3.  In  order  to  use 
temporally  and  spatially  modulated  signals  at  the  system  input  the  electro-optic  modulator 
has  been  replaced  by  a  nematic  liquid  ciystal  spatial  light  modulator  (SLM)  taken  from  a  TV 
projector.  It  was  possible  to  get  oscillation  in  the  resonator  when  a  short  video  movie  was 
played  at  the  input.  However,  only  poor  discrimination  among  similar  movies  could  be 
achieved.  The  reason  for  that  was  the  difficulty  to  obtain  zero-mean  input  signals  using  the 
available  SLM’s.  Non  zero-mean  signals  tend  to  be  characterized  by  their  large  zero  order 
component.  By  choosing  a  proper  configuration  we  were  successful  in  constructing  zero- 
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mean  signals  using  conventional  SLM’s,  however  at  the  costs  of  an  unaffordable  waste  of 
light.  In  contrast,  spatially  and  temporal  modulated  zero-mean  signals  with  large  light 
intensity  can  be  easily  obtained  when  a  discrete  small  number  of  independently  time- 
modulated  Gaussian  beams  are  used  instead  of  a  SLM.  Excellent  signal  discrimination 
could  be  obtained  using  two  such  beams  for  pumping  a  system  similar  to  the  one  in  Fig.  3. 
This  approach  is  relevant  for  potential  system  expansions  through  resonator  cascading. 

5.  Conclusions 

In  the  final  phase  of  the  present  research  project  we  have  developed  a  self-organizing 
optical  processor  which  can  extract  meaningful  temporal  features  from  its  input.  The 
system  makes  use  of  the  delay-line  device  developed  earlier  [8].  Like  the  previously 
investigated  network  for  acoustic  signal  recognition  [3],  also  the  present  system  functions 
through  the  interaction  of  short-  and  long-term  memories.  However,  the  imbedding  of  the 
two  elements  in  a  photorefractive  resonator  makes  the  current  system  suitable  for 
unsupervised  learning.  We  have  experimentally  demonstrated  the  extraction  of  both  binary 
and  analog  features,  as  well  as  the  separation  of  two  features  into  different  outputs. 

In  a  Morse  signal  the  letters  of  the  alphabet  are  comprised  of  a  short  sequence  of  basic 
Morse  features;  words  are  comprised  of  a  sequence  of  letters,  and  so  on.  Similarly, 
complex  temporal  signals  such  as  speech  or  music  are  characterized  by  a  given  hierarchy  of 
short  and  long  temporal  features,  where  longer  features  are  comprised  of  the  basic  (short) 
ones.  In  this  work  we  have  developed  a  processor  which  detects  the  most  basic  features. 
The  unsupervised  identification  of  complex  signals  should  be  possible  using  a  cascaded 
configuration  of  many  such  processors. 
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Schematic  of  an  optical  resonator  incorporating  a  delay  line  for 
learning  temporal  information.  The  collection  of  coupled  rings  forms 
a  chronomode. 
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Fig.2:  Schematic  of  two  competing  resonators.  Each  resonator  ring  learns 
to  be  associated  with  one  of  the  temporal  features.  The  corresponding 
chronomode  responds  only  when  that  feature  is  present  at  the  input 
(lower  plots). 


Fig.  3:  Experimental  Set-up.  Resonator  oscillation  is  between  mirror  Ml  and 
the  rotating  crystal  BaliOS  #1.  Two  chronomodes  are  identified  by  the 
dashed  and  dotted  lines. 
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Fig.  4:  Steady  state  resonator  response  after  presentation  with  two  binary  sub-signals 
SI  and  S2.  SI  =(+-+-+-+-+-+-+-+-),  S2=(4-i-i~— H-+++ — ).  a)  SI  dominant,  it 
occurs  twice  more  often  than  S2,  the  resonator  learns  SI.  b)  S2  is  dominant,  the 
resonator  learns  S2.  The  lower  traces  indicate  which  sub-signal  is  present  at  the 
input  at  a  given  time.  The  16  bit  sequences  begin  at  the  left  edges  and  end  at  the 
right  edges  of  the  square  pulses. 


Fig.  5:  Self-organized  separation  of  two  analog  temporal  sub-signals  into 
two  chronomodes.  (a)  Modulation  of  the  input  signals  1  and  2.  The  two  signals 
are  presented  alternately,  (b)  Istantaneous  profile  of  the  two  chronomodes  in 
response  to  Signal  1.  (c)  Same  in  response  to  Signal  2.  The  lower  chronomode 
has  learned  to  be  associated  with  Signal  1,  the  upper  with  Signal  2. 


